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1.0  INTRODUCTION 

Vector  Research,  Incorporated,  (VRI)  performed  a  small  study  for  the 
Army  Research  Institute  (ARI)  to  assist  in  identifying  human  factors 
research  areas  in  command  and  control  (hereafter  referred  to  as  HF/C2) 
of  importance  to  the  Army.  This  objective  was  accomplished  by  performing 
three  tasks: 

(1)  reviewing  related  HF/C2  literature, 

(2)  identifying  approaches  to  evaluate  C2  activities/systems, 
and 

(3)  developing  a  framework  which  could  be  used  to  identify 
important  areas  of  HF/C2  research, 

with  primary  emphasis  on  the  third  one.  This  report  documents  the 
results  of  the  study. 

The  report  is  comprised  of  six  sections,  including  this  introductory 
one,  and  an  appendix.  Section  2.0  discusses  the  role  and  activities  of 
behavioral  analysis/science  in  Army  planning  and  operations  since  it  is 
these  activities  which  (should)  motivate  ARI's  research  programs  in 
general  and  the  HF/C2  ones  in  particular.  A  principal  activity  in 
identifying  HF/C2  research  areas  was  the  structuring  of  a  framework  or 
paradigm  which  highlights  dimensions  relevant  to  the  development  and  use 
of  HF/C2  information  for  the  Army.  The  dimensions  of  such  a  framework 
are  described  in  section  3.0.  We  believe  the  framework  will  be  useful, 
not  only  as  a  vehicle  for  continual  identification  of  research  areas,  but 
also  as  a  means  of  demonstrating  the  relevance  of  ARI  programs  to  the 
Army  and  of  providing  underlying  rationale  for  ARI  research  and  analysis 
budgets.  One  of  the  dimensions  --  HF/C2  knowledge  areas  --  is 


>. 
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described  in  more  detail  in  section  4.0,  along  with  initial  subjective 
estimates  of  the  degree  of  HF/C2  knowledge  available.  Section  5.0 
demonstrates  how  the  framework  and  available  HF/C2  knowledge  are  used 
to  identify  potential  ARI  research  efforts  in  HF/C2.  Some  general 
comments  regarding  ARI's  HF/C2  activities  are  given  in  section  6.0. 

The  appendix  discusses  approaches  that  might  be  used  to  evaluate  C2 
activities  and  systems. 

8efore  proceeding  on  this  itinerary,  it  is  useful  to  set  the  scope 
of  the  content  area  considered  in  this  study.  (Section  4.0  details  the 
area  by  delineating  some  of  its  inherent  substance.)  Human  factors  are 
concerned  with  the  application  of  psychological  and  physiological  princi 
pies  to  the  design,  development,  and  use  of  complex  man-machine  systems. 
It  is  taken  to  encompass  substance  considered  under  many  previous  titles 
such  as  human  engineering,  experimental  psychology,  biotechnology, 
psychotechnology,  and  others  created  as  the  discipline  grew  from  the 
major  impetus  of  World  War  II.  HF/C2  is  the  application  of  this  branch 
of  behavioral  science  to  military  "systems"*  that  involve  command  and 
control.2  This  is  broadly  concerned  with  processes  involving  the  col¬ 
lection  of  information,  human  assimilation  of  information,  decision¬ 
making,  and  the  communication/dissemination  of  the  decisions.  Although 
these  processes  are  generally  thought  of  as  occurring  in  C2  systems 

*The  term  system  is  used  in  a  broad  sense  to  include  materiel,  organi¬ 
zation,  and  procedures  necessary  for  an  operational  Army  entity,  recog¬ 
nizing  that  some  systems  are  more  materiel  oriented  and  others  more 
organizational  and  procedural. 

2The  authors  believe  that  there  does  not  exist  a  widely  accepted  and 
operationally  useful  definition  of  command  and  control.  (A  formal  defi¬ 
nition  is  contained  in  JCS  Publication  *1.)  Nor  are  we  foolish  enough 
to  attempt  to  create  one.  We  are  confident  that  readers  interested  in 
this  study  are  aware  of  many  realizations  of  existing  and  planned  C2 
systems  and  systems  that  require  C?. 
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such  as  a  Tactical  Operations  Center  (TOC)  or  the  All  Source  Analysis 
System  (ASAS),  it  should  be  recognized  that  they  also  occur  in  other  Army 
systems  such  as  PATRIOT,  Corps  Support  Weapon  System  (CSWS),  Division-86, 
Pershing  II,  an  XM-1  tank  platoon,  and  the  infantry  squad  which  require 
different  levels  and  degrees  of  command  and  control  to  be  operationally 
effective.1  In  essence,  command  and  control  is  pervasive  throughout 
most  of  the  Army's  systems  and  will  continue  to  be  so  as  new  technologies 
create  the  environment  for  short,  high-attrition  engagements  and 
campaigns  requiring  reduced  spatial  densities  and  rapid  mobility. 


Although  Army  regulations  on  the  subject  are  not  completely 
definitive,  it  appears  that  the  Human  Engineering  Laboratory  (HEL)  is 
concerned  with  the  man-machine  interface  problem  for  operation  of  Army 
systems  and  the  ARI  with  the  organizational  and  procedural  aspects  for 
users  of  such  systems. 
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2.0  ROLE/INFl UENCE  OF  MILITARY  HF/C2 

The  overall  objective  of  military  HF/C2  is  to  enhance  the  perform¬ 
ance  of  command  and  control  in  Arny  systems.  This  is  accomplished  by 
performing  a  number  of  roles  related  to  the  design,  development,  and  use 
of  C2  components  of  systems.  Since  the  degree  to  which  these  roles  are 
performed  by  ARI  behavioral  scientists  strongly  influences  how  well  the 
Army,  per  se,  will  perform,  these  roles  are  sometimes  referred  to  as 

"influences"  in  this  paper.  The  better  the  roles  are  performed,  the  more 

influence  ARI  has  on  helping  the  Army  accomplish  its  missions.  The  pur¬ 
pose  of  this  section  of  the  report  is  to  summarize  the  roles  or  influ¬ 
ences  of  HF/C2  in  the  Army. 

As  noted  above,  the  overall  objective  of  military  HF/C2  is  to 
enhance  performance  of  the  command  and  control  aspects  in  systems.  Thus 
ARI 1 s  HF/C2  programs  should  influence  Army  systems  as  they  proceed 
through  the  acquisition  cycle  which  is  shown  in  exhibit  2-1.  The 
development  of  a  new  system  is  initiated  by  the  preparation  of  a  Mission 

Element  Needs  Statement  (MENS),  a  Letter  of  Agreement  (LOA)  at  the  end  of 

the  conceptual  phase,  and  then  a  Required  Operational  Characteristics 
(ROC)  at  milestone  II.  These  documents  essentially  delineate  objectives 
that  are  to  be  accomplished  with  the  system  which  drive  much  of  the 
remainder  of  the  development,  production  and  deployment  processes.  The 
influence  of  HF/C2  in  this  process  should  begin  in  the  early  conceptual 
phase  and  continue  through  the  system's  development  by  developing  HF/C2 
knowledge,  methods,  and  procedures  and  guiding  their  implementation  to 
accomplish  the  following  activities  in  the  acquisition  process: 


EXHIBIT  2-1:  SYSTEM  ACQUISITION  CYCLE 
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•  HF/C  functional  requirements, 

•  HF/C  functional  specification, 

•  HF/C2  design, 

p 

•  HF/C  performance  evaluation,  and 

p 

•  HF/C  performance  prediction. 

The  MENS,  LOA,  and  ROC  delineate  objectives  to  be  accomplished  by 
the  system.  The  functional  requirements  role  of  HF/C2  is  one  in 
which  the  C2  functions  that  must  be  performed  to  accomplish  the  sys¬ 
tem's  objectives  are  identified. *  Additionally,  the  inputs,  processes, 
outputs,  and  required  performance  standards  for  each  of  the  functions  and 
constituent  tasks  need  be  determined  for  each  component  of  the  system. 

In  essence,  this  role  influences  what  C2  functions  need  to  be 
performed. 

In  the  functional  specification  role,  alternative  means  of 
accomplishing  each  of  the  C2  functions,  and  tasks  are  broadly  evaluated 
leading  to  the  specification  of  who  or  what  is  to  perform  the 
functions,  and  to  some  extent  how  to  perform  them.  Examination  of 
alternatives  considers  performance  of  functions  by: 

•  the  system  itself  or  by  exogenous  entities; 

•  manual,  semi-automated,  or  automated  approaches;  and 

•  single  or  multiple  tasking  of  individuals  or  groups. 

For  example,  order  of  battle  estimates  can  be  organized  by  geographic 
area,  by  intelligence  collection  system,  or  by  type  of  threat  unit.  The 

1 1 n  practice,  this  is  not  an  HF/C2  role,  per  se,  but  rather  one  that 
is  performed  by  operational  personnel,  Derhaps  with  the  assistance  of 
behavioral  scientists  to  provide  guidance  on  feasibility  of  achieving 
the  requirements. 
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evaluation  and  selection  of  particular  approaches  considers  broad 
estimates  of  associated  resource  requirements  (funds  and  personnel), 
feasibility  of  achieving  required  performance  standards,  the  effects  that 
operational  environments  may  have  on  these  evaluation  factors,  and 
requirements  for  redundancy. 

Given  the  functional  requirements  and  specification  of  the  approach 
to  be  used  to  achieve  them,  a  major  role  of  HF/C2  is  to  influence  the 
design  of  principal  C2  components  of  the  system.  This  includes 
influencing  detailed  design  of  the  C2  (or  C2  related)  tasks  to  be 
performed  by  individuals  or  groups  of  individuals;  design  of  the  organ¬ 
izations)  necessary  to  operate  the  system;  design  of  C2  mater¬ 
iel  1  or  materiel  aids  intended  to  support  the  C2  function;  and  the 
design  of  efficient  individual  and  organizational  training  to  insure 
effective  C2  when  the  system  is  deployed. 

The  role  of  performance  evaluation  is  concerned  with  measuring 
the  degree  of  individual  proficiency  and  overall  effectiveness  of  the 
C2  function  associated  with  the  system.  Performance  prediction  is 
concerned  with  testing  the  potential  of  individuals  and  groups  of  indi¬ 
viduals  to  accomplish  required  C2  tasks,  and  the  selection  of  system 
personnel  based  on  the  test  results.  Recognizing  that  evaluation  and 
prediction  are  done  by  other  components  of  the  Army,  the  role  of  HF/C2 
is  to  design  instruments  for  evaluation  and  prediction  and  provide  guid¬ 
ance  on  their  use. 

Conceptually,  these  roles  or  influences  of  HF/C2  occur  at  differ¬ 
ent  phases  of  a  system's  acquisition  as  indicated  in  exhibit  2-2. 


*A  significant  portion  of  the  human  factors  guidance  on  material  design 
and  man-machine  interfaces  is  performed  by  the  Army's  Human  Engineering 
laboratory . 
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We  have  taken  the  liberty  of  referring  to  the  validation  and  full  scale 
development  phases  as  the  "development"  phase.  Although  this  exhibit 
reflects  sequential  HF/C^  influences  on  a  system's  development,  in 
practice  there  is  (or  should  be)  continual  feedback  and  interaction  amonq 
them.  Additionally  the  relationship  reflected  in  exhibit  2-3  suggests 
that  the  HF/C^  influence  should  occur  early  in  a  system's  development 
since  decisions  made  by  the  completion  of  Acquisition  Milestone  I  appear 
to  have  a  significant  impact  on  the  system's  life  cycle  costs.  The  fact 
that  (a)  the  Army  always  has  many  systems  in  each  of  the  acquisition 
cycle  phases,  and  (b)  a  system's  effectiveness  is  highly  dependent  on 
achieving  good  performance  (be  it  a  weapon  system,  a  support  system, 
or  a  system  per  se),  strongly  suggests  that  HF/C^  efforts  can  and 
should  have  a  pervasive  influence  on  the  Army. 


EXHIBIT  2-3:  LIFE  CYCLE  COST  DECISIONS 
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3.0  HF/C2  FRAMEWORK 


This  section  of  the  report  outlines  a  framework  for  determining 
human  factors  research  and  analysis  needs  in  the  area  of  command  and  con¬ 
trol.  It  Is  a  systems -oriented  framework  to  depict  where  the  HF/C2 
research  programs  have  an  influence  on  the  Army,  and  to  suggest  that 
ARI's  research  program  should  be  developed  with  a  systems  perspective  to 
insure  its  relevance.  This  is  not  to  suggest  that  the  total  ARI  HF/C2 
program  must  be  directly  tied  to  Army  systems,  but  rather  that  a 
significant  part  must  have  this  orientation.  Clearly,  some  aspects  of 
the  program  should  be  non-system  directed  to:  (a)  pursue  Innovative 
behavioral  science  technologies;  (b)  anticipate  need  of  HF/C2  roles 
further  downstream;  and  (c)  integrate  and  document  HF/C2  knowledge, 
methods,  procedures,  etc.  for  widespread  use  by  the  Army  after  they  have 
been  tested  and  successfully  demonstrated  by  the  ARI. 

The  framework  is  developed  by  defining  dimensions  that  should  be 
considered  in  identifying  relevant  HF/C2  efforts  of  use  to  the  Army. 

The  first  dimension  Is  the  HF/C2  Role  discussed  in  the  previous 
section.  For  completeness,  the  roles  are  repeated  below: 

•  HF/C2  functional  requirements 

•  HF/C2  functional  specification 

•  HF/  C2  design 

•  tasks 

•  organizations 

•  C2  materiel 
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•  HF/C^  performance  evaluation 

•  HF/C^  performance  prediction 

The  next  dimension  in  the  framework  is  the  HF/C^  Knowledge 
that  is  required  to  perform  the  HF/C^  roles.  Knowledge  is  defined  as 
an  understanding  of  the  relationship  between  the  degree  to  which  a  role 
Is  performed  and  the  many  factors  (referred  to  as  knowledge  factors) 
which  influence  it.  As  an  example,  consider  the  role  of  designing  the 
analysis/integration  task  performed  by  humans  in  an  information 
processing  center. 1  Information  useful  in  performing  this  role  would 
be  knowlege  of  the  relationships  between  measures  of  analysis/integration 
task  performance  and  the  following  knowledge  factors: 

Cognitive  factors: 

•  data  characteristics 

•  types  of  decision  processes  and  criteria  employed 

•  perceptual  and  conceptual  skills 

•  task  loading 

Work  Station  factors: 

•  data  base  size  and  content 

•  data  entry  modes 

•  data  retrieval  modes 

•  type  of  data  display  (medium,  format,  coding) 

•  decision  aids  for  prompting,  querying,  processing 

•  general  work  station  characteristics 


^Assuming  the  decision  was  made  to  perform  this  task  by  humans  during 
the  functional  specification  role  for  the  system. 
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Organizational  factors; 

•  si  ze 

•  functional  responsibility  assignments 

•  span  of  control 

•  information  flow 
Training  factors: 

•  kind  of  training  anticipated  (initial /maintenance,  individual/ 
unit) 

•  mode  of  training  (classroom,  simulation,  field,  etc.) 

•  frequency 

•  duration 
Environmental  factors: 

•  work  station  (noise,  temperature,  lighting) 

•  shock  and  vibration  (movement  related) 

Although  this  example  was  Intended  to  reflect  knowledge  requirements  to 
*  . 

design  a  task  (analysis/integration),  the  same  knowledge  would  be 
useful  to  guide  design  of  the  materiel /human  interface  to  perform 
the  task,  training  for  the  task,  and  the  organizational  aspects  related 
to  the  task  since  these  are  explicit  factors  in  the  rel ationship.l 

Knowledge  of  this  sort  is  needed  for  behavioral  scientists  to  per¬ 
form  their  HF/C^  roles  in  the  Army's  system  acquisition  process, 
analogous  to  the  engineer's  use  of  physical  science  relations  to  select 
and  design  the  materiel  aspects  of  systems.  Section  4.0  of  this  report 

*It  Is  true  that  additional  knowledge  would  be  required  to  design  an 
effective  organization  which  encompassed  many  C*  tasks  and  functions. 
Additional  organizational  factors  that  would  have  to  be  considered 
Include  the  number  of  primary  and  support  tasks  involved,  coordination 
requirements  (frequency,  duration,  etc.),  mobility  requirements, 
redundancy,  etc. 
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delineates  a  spectrum  of  relevant  knowledge  areas  for  the  HF/C2  analyst 
and  provides  initial  subjective  estimates  of  the  degree  of  HF/C2 
knowledge  available. 

The  Development  Status  of  Systems  is  the  third  dimension  of  the 
HF/C2  framework.  A  system  can  be  in  either  a  conceptual,  development 
(validation,  fullscale),  or  production/operational  status,  somewhat  anal¬ 
ogous  to  the  acquisition  phases.  As  will  be  shown  later  in  the  report, 
this  dimension  provides  the  means  to  indicate  the  relevance  of  ARI's 
HF/C2  programs  to  specific  system  developments.  The  HF/C2  roles 
required  and  when  in  the  development  cycle  the  roles  must  be  performed 
are  identified  for  each  system  requiring  HF/C2  support. 

Fxhibit  3-1  lists  the  status  of  a  number  of  Army  battlefield 
automated  systems  by  function  type,  and  some  example  estimates  of  the 
kind  of  HF/C2  role(s)  needed  to  support  them  and  approximate  times  that 
the  support  must  be  be  provided.1  Although  ARI's  responsibility 
encompasses  procedural  and  organizational  HF/C2  issues  in  all  Army 
systems,  battlefield  automated  systems  is  used  in  this  example  to  portray 
the  requisite  information  pairing  between  an  Army  system’s  status  and  the 
HF/C2  role  because  ARI's  current  emphasis  is  on  HF/C2  in  automated 
systems.  However,  the  development  of  ARI  HF/C2  research  and  analysis 
programs  should  explicitly  consider  HF/C2  issues  in  manual  and 
semi -automated  systems  (such  as  Patriot,  Pershing  II,  CSWS2)  and 
organizational  ones  (such  as  Division-86,  the  Fire  Integration  Support 
Team,  CORPS-86,  the  Infantry  squad,  etc.).  Additionally,  because  of  the 


JTlmes  an;  listed  where  the  estimate  was  deemed  reasonable.  An  x  is 
used  to  Indicate  that  the  role  needs  to  be  performed,  but  the  time  of 
performing  it  is  uncertain. 

2Some  of  these  HF/C?  activities  may  be  performed  by  the  appropriate 
ARI  field  unit. 
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impact  of  early  decisions  on  life  cycle  costs  reflected  in  exhibit  2-3, 
the  HF/C2  program  also  should  be  designed  to  have  influence  on 
pre-concept  phase  system  candidates. 

The  fourth  dimension  of  the  HF/C2  framework  is  Type  of  ARI 
HF/C2  Effort.  Categories  of  this  dimension  are: 

•  Application  Demonstration  -  the  use  of  well-known  HF/C2 
knowledge,  methods,  etc.  to  demonstrate  how  they  are  to  be 
implemented  in  developing  a  system  or  system  type.  These  are 
essentially  6.3a  or  6.3b  ROTE  category  efforts. *  An  example 
might  be  the  use  of  span-of-control  limits  to  assist  in  designing 
the  organization  of  an  ASAS. 

•  Applied  Research  -  the  application  of  general  HF/C2 
knowledge  to  a  specific  system.  In  contrast  to  ohysical  science 
relationships  which  usually  can  be  applied  directly  in  system 
design,  most  behavioral  science  relationships  are  thought  of  as 
hypotheses  to  be  tested  for  applicability  to  specific  system 
designs.  An  example  might  be  the  design  of  displays  for  an  ASAS 
type  system  which  would  require  an  experimental  project  to 
ascertain  if  general  hypotheses  regarding  human/display 
interactions  were  valid  in  the  context  of  an  ASAS  environment. 
This  type  of  ARI  effort  will  usually  be  classified  in  the  6.3a 
ROTE  program  category. 


*The  ROTE  category  of  the  ARI  effort  need  not  be  related  to  the  phase 
(i.e.,  conceptual,  validation,  etc.)  and  related  program  category  (i.e., 
6.1,  6.2,  6.3a,  6.3b,  6.4,  etc.)  of  the  system  to  which  the  results  of 
the  ARI  effort  are  eventually  applied.  Thus,  for  example,  an  ARI  6.3a 
effort  may  be  undertaken  to  support  systems  in  the  pre-  or  early 
conceptual  phase  and  an  ARI  6.1  basic  research  effort  may  be  undertaken 
to  support  systems  in  production  or  deployed. 
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•  Exploratory  Research  -  the  development  of  general  HF/C2 
knowledge.  Examples  include  the  determination  of  research 
instruments  to  do  some  of  the  applied  research*  and  a  project 
to  develop  knowledge  that  would  be  useful  in  training  humans  to 
do  effective  order-of-battle  intelligence  processing.  This  type 
of  effort  will  usually  be  classified  in  the  6.2  RDTE  program 
category.2 

The  last  dimension  of  the  HF/C2  framework  is  the  First-Use  Time 
Horizon.  This  is  the  time  period  before  results  (adequate  knowledge) 
of  a  proposed  or  in-being  AR1  HF/C2  project  or  program  first  would  be 
used  to  support  the  development  or  operation  of  an  Army  system.  Three 
categories  of  this  dimension  are  deemed  appropriate: 

•  Immediate:  first  use  in  0-3  years 

•  Midterm:  first  use  in  3-5  years 

•  Long  term:  first  use  in  5-10  years 

Although  not  immediately  apparent,  the  cartesian  product  of  Type  of  ARI 
Effort  and  First-Use  Time  Horizon  dimensions  are  possible,  and  not  just 
the  obvious  combinations.  An  exploratory  research  project  could  produce 
information  regarding  some  C2  function  that  might  be  used  immediately 


*This  example  highlights  the  fact  that  some  of  the  basic  HF/C2 
research  is  not  for  the  purpose  of  developing  knowledge  directly,  but 
rather  to  develop  methods/instruments  for  the  knowledge  research. 

2As  noted  at  the  beginning  of  this  section,  some  aspects  of  teh  ARI 
program  will  be  non-system  specific.  Some  of  this  non-system  oriented 
work  will  have  a  less  obvious  link  to  HF/C2  issues  (e.g.,  researching 
innovative  behavioral  science  technologies  to  explore  potential  utility 
for  Army  HF/C2)  and  accordingly,  should  be  considered  basic  research 
in  the  6-1  RDTE  program  category.  (The  distinction  between  6.1  and 
early  6.2  efforts  in  DoD  research  organizations  is  at  times  not  clear.) 
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(0-3  years)  to  suggest  that  the  C2  function  be  performed  automatically 
in  structuring  a  new  system  that  is  still  in  the  conceptual  phase.  In  a 
similar  fashion,  an  applied  or  exploratory  research  project  could 
generate  organizational  knowledge  that  could  be  used  immediately  to  alter 
the  organization  of  an  existing  system  to  improve  its  effectiveness. 

In  summary,  the  framework  for  identifying  human  factors  research 
needs  in  the  coimand  and  control  area  is  comprised  of  five  dimensions  as 
shown  in  exhibit  3-2.  It  is  a  systems-oriented  framework  to  depict  where 
the  HF/C2  research  programs  have  an  influence  on  the  Army,  and  to 
suggest  that  ARI's  HF/C2  program  should  be  developed  with  a  system's 
perspective  to  insure  its  relevance.  Program  development  would  follow 
the  logic  suggested  in  exhibit  3-2.  ARI  would  continually  track  the 
status  of  Army  systems  requiring  HF/C2  support  and  identify  specific 
HF/C2  roles  to  be  performed.  HF/C2  knowledge  requirements  to  provide 
the  support  would  be  identified  by  analysis  of  the  systems  and  the 
HF/C2  roles.  Comparison  of  these  knowledge  requirements  with  existing 
HF/C2  knowledge  would  indicate  the  types  of  analysis/research  efforts 
that  might  be  included  in  ARI's  HF/C2  program. *  The  process  of 
working  through  this  paradigm  to  develop  potential  projects  for  ARI's 
HF/C2  research  and  analysis  program  is  demonstrated  in  section  5.0. 

The  next  section  of  the  report  expands  the  HF/C2  knowledge  area 
dimension  and  provides  an  initial  estimate  of  the  degree  of  such 
knowledge  available  for  use  in  providing  the  Army  with  HF/C2  support. 

1  The  actual  analysis/research  program  (by  types  of  ARI  effort 
required)  would  be  developed  with  consideration  of  DA  funding 
guidelines,  system  priorities,  and  the  desired  amount  of  non-systems- 
oriented  research. 


i 
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EXHIBIT  3-2:  FRAMEWORK  FOR  IDENTIFYING  HF/C2  RESEARCH  NEEDS 
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4.0  KNOWLEDGE  AREAS  IN  HF/C2 

The  preceding  sections  of  this  report  have  outlined  a  five  dimen¬ 
sional  framework  for  determining  human  factors  research  and  analysis 
needs  in  the  area  of  command  and  control.  This  section  discusses  one  of 
those  dimensions,  namely  HF/C^  knowledge,  and  is  intended  to  indicate 
examples  of  the  kinds  of  knowledge  we  believe  necessary  for  effective 
performance  of  the  different  HF/C^  roles  described  in  Section  3.0  and 
to  provide  a  preliminary  estimate  of  the  availability  of  HF/C^  knowledge. 

HF/C^  knowledge  was  defined  to  be  an  understanding  of  the  relation¬ 
ship  between  the  degree  to  which  a  role  is  performed,  and  the  many  factors 
which  influence  it.  In  order  to  examine  knowledge  and  factors  in  the 
context  of  command  and  control,  we  first  propose  a  functional  paradigm  of 
the  command  and  control  process,  that  is,  a  decomposition  of  a  generic 
command  and  control  process  into  functions  and  the  tasks  which  coiimrise 
those  functions.  We  then  discuss  knowledge  and  knowledge  factors  rel event 
to  each  of  the  tasks  included  in  the  paradigm  and  include  an  estimate  of 
the  extent  of  knowledge  currently  available. 

4.1  COMMAND  AND  CONTROL:  A  FUNCTIONAL  PARADIGM 

As  noted  earlier  in  this  report,  there  is  no  commonly  accepted 
definition  of  the  command  and  control  process,  nor  is  there  an  agreed 
upon  description  of  the  components  of  the  process.  Broadly  speaking, 
command  and  control  is  concerned  with  the  collection  of  information, 
human  assimilation  of  information,  decision  making,  and  the  communication/ 
dissemination  of  the  decisions:  elements  which  are  present  in  varying 
degrees  in  all  Army  systems.  A  variety  of  functional  descriptions  of  the 


generic  command  and  control  process  have  been  developed.  One  such 
description  is  illustrated  in  exhibit  4-1,  based  on  research  carried  out 
to  develop  representations  of  command  and  control  in  models  of  combat. 

As  illustrated  in  the  exhibit,  the  command  and  control  process  is  assumed 
to  consist  of  three  major  functions: 

(1)  Information  Management; 

(2)  Information  Processing;  and 

(3)  Action  Selection  and  Implementation. 

Information  management  encompasses  the  acquisition  of  sensed  or  pro¬ 
cessed  data  by  the  command  and  control  process,  the  initial  evaluation  or 
validity  assessment  of  this  data,  and  the  storage  and  management  of  data 
in  a  generic  data  base  or  memory.  Depending  upon  the  level  of  the  com¬ 
mand  and  control  process  under  consideration,  information  management 
could  involve  tasks  ranging  from  a  visual  search  of  a  portion  of  the  bat¬ 
tlefield  to  the  employment  of  a  division  CEWI  battalion,  or  the  assess¬ 
ment  of  input  data  ranging  from  a  blip  on  a  radar  screen  to  an  intelli¬ 
gence  summary  from  a  subordinate  brigade.  The  data  base  or  memory  in 
question  could  range  from  a  hard  copy  intelligence  journal  with  acetate 
overlays  to  an  automated  data  base  system  with  CRT  displays. 

Information  processing  is  viewed  as  consisting  of  five  major 
tasks.  Analysis  and  integration  is  defined  to  be  that  task  in  which 
input  data  is  correlated  and  fused  with  existing  data  to  update  the 
generic  data  base  or  memory.  Perception  update  is  the  task  in  which 
processed  data  is  incorporated  into  an  updated  perception  of  the  battle¬ 
field:  enemy  forces,  friendly  forces,  and  the  environment.  Situation 
assessment  is  defined  to  be  that  task  in  which  the  dispositions,  quanti¬ 
ties  and  activities  of  friendly  and  enemy  forces,  as  perceived,  are 
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examined  and  projected  into  the  future  to  identify  requirements  for  new 
or  revised  plans,  operations,  or  actions.  Construction  of  alternatives 
is  the  task  in  which  alternative  courses  of  action  are  developed  and 
specified  as  a  consequence  of  requirements  identified  in  the  projection 
of  the  situation.  The  final  task  of  information  processing  is  the 
projection  of  consequences,  in  which  the  consequences  of  adopting  any 
specified  alternative  are  estimated  by  consideration  of  potential  enemy 
activities  and  realizations  of  natural  environments. 

The  third  major  function  of  the  command  and  control  process  involves 
the  selection  of  an  action  and  subsequent  implementation  of  the  proce¬ 
dures  by  which  subordinates  are  tasked,  other  entities  are  informed 
and/or  activities  are  initiated.  Selection  of  an  action  involves  the 
application  of  criteria  to  the  consequences  developed  for  different 
alternatives  and  includes  both  the  choice  of  a  null  action,  (i.e.,  a 
decision  not  to  decide  or  act)  and  a  recognition  of  a  requirement  for 
further  development  of  alternatives,  (i.e.,  further  information  process¬ 
ing).  Implementation  of  a  selected  action  includes  the  development  and 
communication  of  plans  and  orders  throughout  the  command  hierarchy.  At 
the  lowest  levels  implementation  could  be  as  simple  as  aiming  and  Tiring 
a  weapon. 

Dynamically,  the  process  composed  of  the  functions  and  tasks  defined 
above  should  be  viewed  as  operating  continuously,  with  many  functions  and 
tasks  carried  out  in  parallel,  rather  than  in  series.  Furthermore,  as 
will  be  evident  below  in  discussions  of  HF/C^  knowledge,  the  distinc¬ 
tions  between  the  various  tasks  can  become  blurred  depending  upon  the 
numbers  and  responsibilities  of  personnel  performing  command  and  control. 


Nonetheless,  the  paradigm  does  provide  a  basis  for  organizing  the 
discussion  of  C2  knowledge  and  knowledge  factors. 

4.2  HF/C2  KNOWLEDGE  AND  KNOWLEDGE  FACTORS 

As  noted  previously  in  this  report,  HF/C2  influences  the  develop¬ 
ment  and  operation  of  any  system  through  performance  of  five  roles: 

(1)  HF/C2  functional  requirements; 

(2)  HF/C2  functional  specification; 

(3)  HF/C2  design: 

(a)  tasks; 

(b)  organizations; 

(c)  C2  materiel ; 

(d)  training; 

(4)  HF/C2  performance  evaluation;  and 

(5)  HF/C2  performance  prediction. 

Effective  performance  of  the  five  HF/C2  roles  depends  upon  the  extent 
of  available  HF/C2  knowledge.  HF/C2  knowledge  is  defined  to  be 
information  regarding  the  relationship  between  degree  to  which  relevent 
knowledge  factors  and  the  C2  functions  and  tasks  shown  in  exhibit  4-1 
can  be  performed.  The  remainder  of  this  section  consists  of  a  discussion 
of  knowledge  and  knowledge  factors  required  in  the  application  of  HF/C2 
to  the  development  and  operation  of  systems  which  require  command  and 
control.  The  discussion  will  be  based  on  the  command  and  control 
paradigm  presented  above.  Since  knowledge  to  assist  in  performing  the 
design  role  (tasks,  organizational,  C2  materiel,  and  training)  also 
provides  knowledge  to  assist  in  the  functional  requirements  and  speci¬ 
fication  roles,  these  are  considered  simultaneously.  The  roles  of 
performance  evaluation  and  prediction  are  considered  separately. 


4.2.1  FUNCTIONAL  REQUIREMENTS,  FUNCTIONAL  SPECIFICATIONS  AND  DESIGN 
KNOWLEDGE  FACTORS 

In  a  broad  sense,  there  are  five  principal  classes  of  HF/C2  know¬ 
ledge  factors  that  influence  the  functional  requirements,  functional 
specifications  and  design  HF/C2  roles  in  the  development  of  any 
system: 

(1)  Cognitive; 

(2)  Work  Station; 

(3)  Organizational; 

(4)  Training;  and 

(51  Environmental. 

Cognitive  factors  fall  into  three  principal  categories.  The  first 
encompasses  the  individual's  knowledge  base,  the  types  and  sources  of 
input  required  by  the  function  or  task  under  consideration  and  the 
outputs  produced,  including  the  effects  due  to  the  dynamics  of  combat. 
The  second  category  includes  personnel  attributes  associated  with  per¬ 
ceptual  processes  and  skills,  and  the  third  involves  the  decision 
processes  utilized  internally  in  task  performance. 

Work  Station  factors  are  associated  with  man/machine  interfaces  and 
overall  user/operator  work  station  task  procedures.  In  HF/C2  analyses, 
these  factors  are  related  to  system  supports  such  as  data  base  or  memory 
enhancement  and  decision  aids,  and  thus  to  factors  such  as  data  entry, 
data  display,  and  data  retrieval,  in  the  context  of  the  particular 
operator  work  station  under  consideration.  Work  station  factors  fall 
into  two  categories:  user  procedures  which  are  usually  considered  to  be 
ARI's  responsibility,  and  hardware  design  which  is  usually  considered  to 
be  HEL's  responsibility. 
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Organizational  factors  include  functional  task  assignments  (both 
supervisory  and  production)  to  individuals,  the  specification  of  respon¬ 
sibility  (span  of  control),  information  flow,  and  management  procedures, 
and  sizing.  Combat  imposes  constraints  and  requirements  on  the  organiza¬ 
tion  of  all  units,  and  comand  and  control  organizations  thus  must 
include  redundancy  and  be  versatile  to  ensure  adequate  performance  when 
damaged,  dispersed,  or  severely  stressed. 

Training  factors  are  those  associated  with  establishing  and  main¬ 
taining  required  performance  levels  for  personnel  and  organizations. 
Performance  deficiencies  in  existing  systems  may  be  due  to  lack  of  profi¬ 
ciency,  rather  than  lack  of  capability  and  as  such  are  the  province  of 
the  HF/C2  analyst.  Given  any  particular  system  the  degree  to  which  it 
accomplishes  its  objectives  will  depend  upon  the  degree  to  which  person¬ 
nel  have  acquired  and  maintained  proficiency  in  required  skills,  i.e., 
the  degree  to  which  training  has  been  effective.  Training  factors  thus 
consider  such  topics  as  initial  training  (skill  acquisition)  and  profi¬ 
ciency  retention  training,  individual  and  organizational  training,  fre¬ 
quency,  duration,  and  mode  (e.g.,  classroom,  simulation,  field  exercise, 
etc.).  Consideration  of  these  factors  is  significant  in  the  early  stages 
of  system  development  because  of  impact  on  life  cycle  cost  and  manning 
policies. 

Environmental  factors1  are  those  associated  with  the  impact  of  the 
physical  environment  on  human  performance,  such  as  temperature,  noise, 

significant  portion  of  research  in  environmental  factors  is  carried 
out  by  HEL  and  the  Surgeon  General. 


lighting,  etc.  To  some  extent  these  factors  can  be  controlled  by  appro¬ 
priate  operator/work  station  design,  however,  battlefield  conditions  may 
exceed  design  boundaries. 

4.2.2  PERFORMANCE  EVALUATION  AND  PREDICTION  KNOWLEDGE  FACTORS 

As  discussed  in  Section  3.0  of  this  report,  in  addition  to  perform¬ 
ing  HF/C2  roles  in  Functional  Requirements ,  Functional  Specification 
and  Design,  the  HF/C2  analyst  also  performs  roles  in  Performance 
Evaluation  and  Performance  Prediction.  These  roles  are  performed  in  the 
latter  stages  of  system  development  and  deployment,  and  the  degree  to 
which  they  are  accomplished  successfully  depends  to  a  large  extent  on  the 
degree  to  which  cognitive  aspects  of  tasks  and  task  objectives  have  been 
defined  during  system  design. 

Performance  evaluation  is  carried  out  to  determine  the  degree  to 
whi ’h  an  individual  or  organization  will  meet  established  standards  of 
performance.  Associated  factors  include  the  nature  and  performance 
standards  of  the  task,  consideration  of  differences  between  anticipated 
operational  environment  and  evaluation  environment,  the  entity  evaluated 
(individual  or  organization)  and  the  characteristics  of  the  evaluation 
instrument. 

Performance  prediction  factors  are  related  to  the  ability  to  predict 
the  degree  to  which  an  individual  is  capable  of  acauiring  a  particular 
skill  or  level  of  performance  prior  to  or  during  training  and  thus 
emphasizes  measurement  of  potential,  i.e.,  in  the  identification  of 
attributes  and  the  relationship  of  attribute  values  to  eventual  perform¬ 
ance  levels.  Performance  prediction  factors  fall  into  two  categories: 
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the  first  pertaining  to  knowledge  of  the  human  factors  attributes 
required  to  perform  a  task;  and  the  second  pertaining  to  prediction 
instruments  which  measure  these  attributes. 

The  remainder  of  this  section  discusses  knowledge  factors  in  the 
context  of  the  command  and  control  functional  paradigm  presented  above. 
Examples  of  cognitive,  work  station,  organi zational ,  training,  and 
environmental  factors  conmon  to  the  command  and  control  tasks  are  listed 
in  exhibit  4-2.  The  knowledge  implied  in  this  exhibit  is  to  assist  in 
performing  the  HF/C2  functional  requi rements ,  functional  specification, 
and  design  roles.  Performance  evaluation  and  performance  prediction 
factors  listed  are  considered  in  exhibits  4-3  and  4-4,  respectively. 

The  knowledge  requirements  and  knowledqe  factors  presented  in 
exhibits  4-2  through  4-4  are  intended  to  indicate  the  kinds  of  informa¬ 
tion  required  by  the  HF/C2  analyst  to  perform  the  HF/C2  roles 
described  in  section  3.0  in  the  context  of  the  tasks  comprising  the 
command  and  control  process.  As  described  in  that  section,  these  are 
used  to  identify  potential  AR I  efforts  (application  demonstration, 
applied  research,  exploratory  research)  to  either  use  or  produce  HF  'C2 
knowledge.  It  is  important  to  recognize,  however,  that  at  times  there 
are  insufficient  methods,  instruments,  and/or  information  available  to 
perform  the  knowledge  generating  research.  Accordingly  methodology 
development  research  may  be  necessary.  For  example,  in  assessing  the 
impact  of  varying  the  quantity  and  quality  of  input  data  on  the  decision¬ 
making  performance  of  a  command  and  control  system,  it  is  first  necessary 
to  specify  the  input  data  used  by  the  system.  This  specification  may 
require  research  to  determine  what  data  is  used  before  the  research  on 
variations  of  input  data  quality  and  quantity  can  be  accomplished. 


EXHIBIT  4-2:  COMMAND  AND  CONTROL  TASKS: 

COMMON  PERTINENT  KNOWLEDGE  FACTORS 


Cognitive  Factors: 

Combat  Dynamics/Status 
Data  Arrival  Rates 
Tasking 


Processing 

Complexity 

Domain 


Work  Station  Factors: 


Data  Base 
Size 
Content 

Data  Entry 

Data  Retrieval 


Data  Display 
Medi urn 
Format 
Coding 

Decision  Aids 
Prompting 
Querying 
Processing 


Personnel 
Perceptual  and 
Cognitive  Skills 
Stress  Thresholds 
Motivation 

Data 

Quantity 
Qual ity 
Variety 


General  Work  Station 
Character! sties 


Organizational  Factors: 
Size 

Span  of  Control 
Information  Flow 
Redundancy 

Training  Factors: 

Initial /Retention 
Individual /Unit 


Task  Assignments 

Supervi sory/Management/Coordi nation 
Production 


Frequency 

Duration 

Mode 


Environmental  Factors: 


Noise 
Humid i t 


Radiation 
L  Tght 


Atmosphere 

Stress 


Temperature 

Shock/Vibration 
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EXHIBIT  4- 


:  PERFORMANCE  EVALUATION:  PERTINENT  KNOWLEDGE  FACTORS 

•  Evaluation  Entity 

•  Indivi dual /Organization 

•  Echelon 

•  C^  Function/Task  Evaluated 

•  C^  Function/Task  Performance  Standards 

•  Function/Task  Complexity 

•  Evaluation  Environment 

vs 

Anticipated  Operational  Environment 

•  Evaluation  Instrument 

•  Type 

•  Written  Tests 

•  Laboratory  Experiments 

•  Simulation 

•  Games 

•  Field  Tests/Exercises 

•  Characteri sties 

•  Reliability 

•  Val idity 

•  Completeness 

•  Evaluators 

•  Objective 

•  Subjective 


EXHIBIT  4-4:  PERFORMANCE  PREDICTION: 


PERTINENT  KNOWLEDGE  FACTORS 


•  Function/Task  Skill  Requirements 

•  Cognitive 

,  Psychological 

•  Physiological 

•  prediction  Instrument 

•  Type 

.  Reliability 

•  Validity 

•  Completeness 
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Similarly,  before  systems  and  procedures  for  performance  evaluation^ 
can  be  developed,  research  may  be  necessary  to  determine  valid  measures 
of  C2  performance.  Thus,  the  factors  presented  are  intended  to  provide 
insight  not  only  into  the  efforts  required  by  ARI  to  effectively 
influence  Army  command  and  control  systems,  but  also  into  the  research 
required  to  maintain  and  create  the  capability  to  accomplish  those 
efforts.  Such  methodology  development  efforts  most  often  will  be  in  the 
exploratory  research  category,  but  may  or  may  not  be  system  oriented. 

4.3  CURRENT  AVAILABILITY  OF  KNOWLEDGE 

The  extent  of  HF/C2  knowledge  available  for  use  in  performing  the 
HF/C2  roles  is  needed  as  a  basis  for  structuring  ARI'S  HF/C2  research 
program.  A  taxonomy  to  categorize  the  degree  of  available  HF/C2 
knowledge  is  described  below. 

(D)  Definitive:  Sufficient  HF/C2  knowledge  of  a  task  (function)  is 
available  to  perform  HF/C2  roles  for  that  task  (function)  for  all 
relevant  system  types  and  operating  environments. 

(S+)  Type  System  Specific:  Sufficient  HF/C2  knowledge  about  a  task 
(function)  is  available  to  perform  HF/C2  roles  for  that  task 
(function)  for  the  specific  system  type  and  operating  environment. 
(S)  System  Specific,  Methodology  Available:  Sufficient  HF/C2 

knowledge  about  a  task  (function)  is  available  to  perform  HF/C2 
roles  for  that  task  (function)  for  a  specific  subset  of  system  types 
and  operating  environments  not  including  those  under  consideration. 
Methodology  is  available  to  extend  that  knowledge  to  the  system 
type  and/or  operating  environments  under  consideration. 

^Evaluation  of  C2  activities  is  discussed  in  the  appendix  to  this 
report. 


(S')  System  Specific,  Methodology  Unavailable:  HF/C2  knowledge  about 
a  task  (function)  is  restrictd  to  a  subset  of  system  types  not 
including  the  type  and  operating  environments  under  consideration. 
Methodology  to  extend  knowledge  to  that  type  and/or  operating 
environments  is  not  available. 

(G+)  General  Knowledge,  Methodology  Available:  The  extent  of  HF/C2 

knowledge  about  a  task  (function)  is  limited  to  generally  accepted 
hypotheses*  evolving  from  general,  non-system  specific  experi¬ 
ments.  Methodology  is  available  to  produce  knowledge  to  the  degree 
required  to  perform  HF/C2  roles,  for  the  specfied  system  type. 

(G“)  General  Knowledge,  Methodology  Unavailable:  The  extent  of  HF/C2 
knowledge  about  a  task  (function)  is  limited  to  generally  accepted 
hypotheses  evolving  from  general,  non-systein  specific  exoeriments. 
Methodology  to  produce  knowledge  to  the  degree  required  to  perform 
HF/C2  roles  for  the  specified  system  type  is  not  available. 

(0+)  Professional  Opinion,  Methodology  Available:  The  extent  of 

HF/C2  knowledge  about  a  task  (function)  is  limited  to  generally 
accepted  professional  opinion.  Methodology  is  available  to  conduct 
non-system  specific  experiments  to  produce  general  or  more  specific 
knowledge. 


*In  constrast  to  physicial  science  relationships  which  usually  can  be 
applied  directly  in  system  design,  most  behavioral  science  relationships 
are  thought  of  as  hypotheses  to  be  tested  for  applicability  to  specific 
system  designs. 
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(0“)  Professional  Opinion,  Methodology  Unavailable:  The  extent  of 
HF/C2  knowledge  about  a  task  (function)  is  limited  to  generally 
accepted,  professional  opinion.  Methodology  does  not  exist  to 
conduct  non-system  specific  experiments. 

(V)  Void:  The  extent  of  HF/C2  knowledge  about  a  task  (function) 
consists  of  conflicting  professional  opinions. 

Exhibit  4-5  summarizes  the  current  status  of  HF/C2  knowledge 
available  to  perform  functional  requi rements,  functional  specification, 
and  design  HF/C2  roles.  This  information  was  obtained  in  work  sessions 
with  ARI  behavioral  scientists  who  have  been  working  in  the  HF/C2  area 
for  many  years.  The  exhibit  is  organized  by  C2  tasks  and  class  of 
knowledge  factor.  The  training  factor  class  has  been  excluded 
deliberately  as  it  was  the  opinion  of  ARI  personnel  that  the  impact  of 
the  training  factor  class,  and  successful  performance  of  the  other 
HF/C2  roles  of  performance  evaluation  and  performance  prediction  are 
all  closely  related  to  the  extent  to  which  task  analyses  can  be  or  have 
been  carried  out,  i.e.,  provided  a  learning  objective  has  been 
identified,  training  programs,  performance  evaluation  instruments,  and 
performance  prediction  methods  can  be  constructed  using  available 
methodology.  It  should  be  noted  that  dependence  among  knowledge  factors 
classes  is  not  restricted  to  the  training  factor  class.  Clearly  the 
successful  design  of  user  work  station  procedures  depends  upon  the  extent 
of  knowledge  of  the  cognitive  factors  as  does  the  incorporation  of 
environmental  factors  and  organizational  issues  into  the  determination  of 
overall  system  characteristics  and  ultimate  performance. 
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EXHIBIT  4-5:  AVAILABILITY  OF  KNOWLEDGE 


\  Knowledge 
\  Factor 
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Cognitive 
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5.0  EXAMPLE  FRAMEWORK  APPLICATION  TO  IDENTIFY  HF/C2 

RESEARCH  EFFORTS 

Section  3.0  of  this  report  presented  a  systems-oriented  framework 
for  identifying  human  factors  research  needs  in  the  command  and  control 
area  in  terms  of  five  dimensions:  System  Development  Status,  HF/C2 
Roles  (Influences),  HF/C2  Knowledge  Areas,  Type  of  ARI  Effort,  and 
First  Use  Time  Horizons.  HF/C2  knowledge  areas  (by  HF/C2  functions, 
tasks  and  knowledge  factors)  and  a  scaling  of  the  degree  of  HF/C2 
knowledge  available  was  presentd  in  section  4.0.  In  this  section  of  the 
report  we  (a)  show  how  the  framework  and  available  HF/C2  knowledge  are 
used  to  identify  ARI  research  efforts  needed  to  support  the  development 
of  any  system  (section  5.1);  and  (b)  demonstrate  the  process  by 
identifying  some  HF/C2  efforts  relevant  to  the  development  of  an  All 
Source  Analysis  System  (ASAS)  (section  5.2).  Since  the  paradigm  can  be 
used  not  only  to  develop  the  HF/C2  program,  but  also  to  reflect  the 
direct  relevance  of  the  program  to  the  Army,  some  alternative  means  of 
presenting  the  research  program  using  the  framework  information  is  given 
in  section  5.3. 

5.1  RESEARCH  PROGRAM  DEVELOPMENT  PROCESS 

The  logic  for  developing  the  systems-oriented  HF/C2  research  needs 
was  schematically  represented  in  exhibit  3-2.  It  is  repeated  in  this 
section  as  exhibit  5-1  as  an  outline  of  the  process  recommended  for  ARI 
to  develop  their  systems-related  HF/C2  research  program.  The  steps  in 
this  process  are  defined  below: 

(1)  ARI  should  continually  track  the  development  status  of  Army 
systems  (materiel,  organizations),  both  those  systems  that  are 
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EXHIBIT  5-1:  FRAMEWORK  FOR  IDENTIFYING  HF/C2  RESEARCH  NEEDS 


DIMENSION 

.stem  Dave  foment 
Status: 


Concaotua  I 
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Development 
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•rlela  Arty 


•Aield  Arty 
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designated  as  formal  Army  programs*  (e.  g. ,  SOTAS,  TACFIRE, 

BCS,  AGTELIS,  TACELIS,  TACJAM,  PATRIOT,  PERSHING  II, 
DIVISION-86)  and  those  that  are  still  in  concept  development 
(e.  g.  ,  ASAS,  TCAC,  CSWS,  ECS^).  It  should  consider  those  in 
concept  development  which  are  advertised  as  moving  rapidly 
through  the  development  cycle  to  an  early  IOC  date  and  those 
pre-natal  developments  that  are  just  being  formulated  by  Army 
laboratories,  with  potential  IOC  dates  10-15  years  downstream. 

(2)  For  major  systems  or  system  types,  ARI  should  identify  what 
HF/C^  Roles  (functional  requi rements ,  functional  specifi¬ 
cations,  design  tasks,  etc.)  remain  to  be  performed  in 
developing  the  system  and  when  in  the  development  cycle  they 
must  be  performed.  (See  exhibit  3-1  for  the  kinds  of  infor¬ 
mation  envisioned  in  this  stage  of  the  process,  although  the 
exhibit  does  not  contain  many  "pre-natal"  systems.) 

(3)  For  each  system-HF/C^  Role  pair  identified,  ARI  should 
determine  specifically  what  must  be  performed  to  accomplish  the 
HF/C“*  Role  for  the  specific  system  (or  system  type)  and  why 
the  activity  is  needed  (i.e.  ,  what  would  happen  if  not 
performed).  Since  the  HF/C^  Role  may  be  performed  by  many 
different  organizations  (e.  g.  ,  industrial  contractors,  TRADOC 
schools,  a  DARCOM  laboratory,  etc.)  ARI  must  further  identify 
what  support  it  will  provide  to  accomplish  the  needed  HF/C^ 
Role.  This  may  vary  from  accomplishing  the  HF/C^  Role 

*For  this  study  we  define  formal  Army  programs  as  those  that  have 
passed  ASARC  I  (or  the  equivalent  stages  for  an  organization's 
development). 
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entirely;  providing  methods,  procedures,  etc.;  providing 
consulting  support;  monitoring  the  effort;  to  no  partici¬ 
pation. 

(4)  For  the  support  that  ARI  intends  to  provide  in  accomplishing 
the  HF/C2  Roles,  it  should  identify  the  knowledge  regarding 
HF/C2  functions,  tasks,  and  related  knowledge  factors* 
required  to  provide  the  support.  The  required  knowledge  should 
then  be  compared  to  the  existing  knowledge?  base  regarding 
the  HF/C2  functions,  tasks,  and  knowledge  factors  to 
determine  the  type  of  ARI  Effort2  needed  to  support  the 
HF/C2  Role.  The  suggested  transformation  to  determine  the 
Type  of  ARI  Effort  needed  is  shown  in  exhibit  5-2.  The 
transformations  reflected  in  the  exhibit  are  based  on  the 
assumption  that  the  required  amount  of  knowledge  available  to 
accomplish  an  HF/C2  Role  increases  as  the  role  to  be 
performed  moves  from  functional  requirements;  to  functional 
specification;  to  design  of  HF/C2  tasks,  organizations,  etc. 

It  is  also  implied  that: 

•  exploratory  research  efforts  are  needed  to  generate  general 
knowledge; 

•  applied  research  efforts  are  needed  to  produce  system 
specific  knowledge;  and 


*See  section  4.0  for  a  discussion  of  HF/C2  functions,  tasks, 
knowledge,  and  knowledge  factors. 

2See  section  4.0  for  a  discussion  of  the  existing  knowledge  categories. 

■*See  section  3.0  for  a  discussion  of  Type  of  ARI  Effort.  Essentially 
these  are  potential  projects  for  ARI's  HF/C2  research  program. 
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EXHIBIT  5-2:  DETERMINING  TYPE  OF  ARI  HF/C2  EFFORT 

CATEGORY  OF  CATEGORY  OF  TYPE  OF  ARI 


HF/C2  ROLE 

REQUIRED  KNOWLEDGE 

EXISTING  KNOWLEDGE 

HF/C2  EFFORT 

Functional 

G+ 

D,S+,S,S-,G+ 

=> 

Application 

Requirements 

Demonstration 

G-,0+ 

=> 

Appl ied 

Research 

o-,v 

=> 

Exploratory 

Research 

Functional 

S 

D,S+,S 

=> 

Appl ication 

Specifications 

Demonstration 

S-,G+,G-,0+ 

=> 

Appl ied 

Research 

o-,v 

=> 

Exploratory 

Research 

Design 

•  Tasks 

S+ 

D,S+ 

=> 

Application 

•  Organizations 

•  Material 

Demonstration 

•  Training 

S,S',  G+,0+ 

=> 

Appl ied 

Research 

G-.O-.V 

=> 

Exploratory 

Research 

Performance 

D,S+ 

=> 

Appl ication 

Evaluation 

Demonstration 

and 

S+ 

S,S" ,G+,0+ 

=> 

Applied 

Research 

Performance 

Prediction 

G-.O-.V 

=> 

Exploratory  Research 

legend:  D  : 

S+: 
S  : 

S": 

G+: 
G": 
0+: 
O': 
V  : 


Definitive  Knowledge 

System  Specific  Knowledge  for  System  Type  under  Consideration 
System  Specific  Knowledge  for  Related  System  Types, 
Methodology  Available 

System  Specific  Knowledge  for  Related  System  Types, 

Methodology  Unavailable 

General  Knowledge,  Methodology  Available 

General  Knowledge,  Methodolooy  Unavailable 
Professional  Opinion,  Methodology  Available 
Professional  Opinion,  Methodology  Unavailable 
Void 


•  application  demonstrations  and  applied  research  efforts  on  a 
large  number  of  systems  are  needed  to  produce  definitive 
knowledge  about  particular  HF/C2  functions,  tasks, 
organizations,  etc. 

(5)  The  specific  methodology  used  (e.g.,  experimental,  survey, 
modeling,  etc.)  in  accomplishing  the  application  demonstra¬ 
tions,  applied  research,  and  exploratory  research  efforts 
derived  in  step  4  will  depend  on  many  factors  including  the 
amount  of  knowledge  shortfall,  costs,  and  schedule  for 
accomplishment  of  the  HF/C2  Role  within  the  system 
development  cycle.  The  latter  is  contained  in  the  systems 
status-HF/C2  Role  information  determined  in  step  2  of  the 
process  and  should  be  used  tc  associate  a  First  Use  Time 
Horizon  with  each  potential  ARI  HF/C2  project. 

In  concluding  this  section,  it  should  be  recognized  that  the  process 
described  would  produce  a  menu  of  potential  projects  for  an  ARI  HF/C2 
research  program.  The  actual  program  (i.e.,  proposed  package  of 
projects)  would  be  developed  with  consideration  of  DA  funding  guidelines, 
system  priorties,  and  amount  of  non- systems  oriented  research  deemed 
necessary. 

5.2  SAMPLE  APPLICATION  OF  THE  PROCESS  FOR  ASAS 

In  this  section  we  demonstrate  the  process  described  above  by 
preparing  a  sample  worksheet  for  HF/C2  research  efforts  related  to 
development  of  an  All  Source  Analysis  System  (ASAS).  As  noted  in  section 
3.0,  the  ASAS  is  a  system  that  is  still  in  concept  development,  although 
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the  Army  is  attempting  to  field  it  in  the  1986-1987  time  period.1 
Exhibits  5-3  and  5-4  are  sample  worksheets  for  projects  which  support  the 
functional  requirements  and  design  HF/C^  Roles  in  development  of  the 
ASAS. 


*At  the  time  of  writing  this  report  an  industrial  contractor  was 
awarded  a  project  to  develop  the  functional  system  design  (i.e., 
functional  specifications)  for  the  system  prior  to  the  end  of  1980. 
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EXHIBIT  5-3:  SAMPLE  WORKSHEET  --  FUNCTIONAL  REQUIREMENTS  ROLE 

PROJECT :  Human  Factors 
THRUST:  Command  and  Control 
RELEVANT  SYSTEM:  ASAS 

REQUIRED  HF/C^  ROLE:  Functional  Requirements 

The  functional  requirements  for  the  All  Source  Analysis  System  are 
influenced  by  two  major  factors: 

(1)  The  necessity  of  fighting  outnumbered  requires  that  the 

commander  be  provided  with  information  that  is  more  timely, 
accurate,  and  extensive  than  that  available  from  the  current 
intelligence  system. 

(21  Innovations  in  sensor  technology  and  capability  have  greatly 
increased  the  amount  of  raw  data  available  to  the  intelligence 
system. 

These  two  factors  lead  to  major  HF/C^  issues: 

(1)  Command  and  control  is  a  complex  process  varying  across 
individuals.  It  is  important  to  delineate  what  information  is 
required,  and  the  accuracy,  timeliness  and  extent  required  of 
that  information  to  respond  to  the  needs  of  commanders. 

(2)  Certain  tasks  (e.g.,  pattern  recognition)  are  performed  well  by 
humans,  while  others  can  be  automated,  or  supported  by 
automation.  Which  tasks  or  functions  in  the  intelligence 
process  should  be  considered  candidates  for  automation 
(replacement  of  manual  implementation  by  ADP)  or 

semi -automation  (support  of  personnel  by  ADP),  and  which  should 
be  performed  manually? 

4 


Continued  - 
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EXHIBIT  5-3:  SAMPLE  WORKSHEET  -  FUNCTIONAL  REQUIREMENTS  RPL E 

(Continued) 

(3)  Automation  and  communication  systems  Drovide  the 
capability  to  increase  information  flow  and  control.  How 
should  the  personnel  and  support  systems  (e.g.,  automated  data 
bases,  decision  aids,  query  features,  etc.)  best  be  organized 
to  produce  required  intelligence,  subject  to  the  constraints 
and  environment  of  military  operations? 

The  functional  requirements  will  be  developed  by  TSM  ,  ASAS  with  support 
provided  by  IBM. 

HF/C2  KNOWLEDGE  REOUIREMENTS/AVAIL ABLE 

The  HF/C2  knowledge  required  to  accomplish  the  functional 
requirements  role  must  fall  into  a  category  of  General  Knowledge, 
Methodology  Available,  or  better.  The  current  availability  of  knowledge 
relative  to  the  ASAS  is  summarized  in  the  following  table.  Comparison  of 
this  table  to  Exhibit  5-2  indicates  that  all  of  those  cells  that  have  an 
entry  of  G+  or  better  implies  that  information  is  available  for  direct 
application  demonstrations.  Cells  indicating  less  knowledge  imply  topics 
or  areas  in  which  applied  or  exploratory  research  efforts  are  needed. 


--  Continued  -- 
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EXHIBIT  5-3:  SAMPLE  WORKSHEET  -  FUNCTIONAL  REQUIREMENTS  ROLE 

(Continued) 


TABLE:  AVAILABILITY  OF  KNOWLEDGE  FOR  ASAS  FUNCTIONAL  REQUIREMENTS 


\i Know!  eage 
Factor 
\C1ass 

c2  \ 

Task  \. 

OJ 

> 

C 

cn 

O 

o 

Work 

Station 

'O 

c 

w 

<T3 

ISJ 

C 

3 

o 

Envi romnental 

Acquisi tion 

. 

G 

+ 

G 

_ 

o+ 

S 

Assessment 

B 

Q+ 

S 

Storage 

G+ 

U 

0* 

s  ■ 

Analysis  and 
Integration 

G 

3 

0' 

s 

Perception 

'JDdate 

'J 

j 

0+ 

s 

Si  tuation 
Assessment 

G+ 

G+ 

0+ 

s 

Construction 

of 

A1  ternati ves 

O' 

G+ 

0' 

s 

Projection  of 
Consequences 

O' 

G+ 

V 

s 

Action 

Sel ection 

G+ 

4> 

G‘ 

G+ 

s 

Implementation 

r*  ** 

r 

o 

5 
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EXHIBIT  5-3:  SAMPLE  WORKSHEET  -  FUNCTIONAL  REQUIREMENTS  ROLE 

(Concl uded) 


TYPE  OF  HF/C2  EFFORT  REQUIRED 

Performance  of  the  functional  requirements  HF/C2  role  requires 
that  Knowledge  fall  in  a  category  of  general  Knowledge  with  methodology 
available  to  extend  the  Knowledge  to  the  system  type  under  consideration. 
Oue  to  the  fact  that  current  determination  of  functional  requirements  is 
being  performed  by  a  contractor,  and  the  fact  that  the  contract  will 
terminate  on  30  September  1980,  all  ARI  participation  in  the  functional 
requirements  HF/C2  role  will  consist  of  monitoring  the  contract  and 
contributing  via  membership  as  an  observer  on  the  Study  Advisory  group. 
This  effort,  an  application  demonstration,  will  center  on  ensuring  that 
available  HF/C2  Knowledge  is  applied  to  the  maxmimum  extent  possible 
and  that  potential  problems  requiring  research  are  identified.  Only  one 
application  demonstration  worK  unit  is  required. 

(1)  Application  Demonstration  WorK  Uni t:XXXXXXX 

Project: _ 

Principal  Investigator: _ 

Start  Date:  1  February  1980 

Objective:  To  ensure  that  HF/C2  Knowledge  is  applied  in  the 

development  of  Functional  Requirements  for  the  ASAS 
and  to  identify  potential  problems  requiring 
HF/C2  research. 


Approach:  Participation  on  ASAS  FSD  SAG  as  an  observer. 

Time  for  First  Use:  1  February  1980 
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EXHIBIT  5-4:  SAMPLE  WORKSHEET  -  DESIGN  ROLE 


PROJECT :  Human  Factors 

THRUST :  Command  and  Control 

RELEVANT  SYSTEM:  ASAS 
REQUIRED  HF/C2  ROLE:  Design 

The  design  of  tasks,  material,  organization  and  training  for  the 
intelligence  analysis  component  in  the  ASAS  system  will  have  a  major 
impact  on  the  performance  of  the  fielded  system.  Alternatives  in  the 
system  currently  in  use  have  been  limited.  However,  automation  provides 
an  opportunity  to  replace  or  support  personnel  involved  in  analysis 
tasks;  furthermore,  this  approach  may  be  necessary  if  increases  in 
available  raw  information  are  to  be  exploited  and  intelligence  needs  met 
in  the  modern  dynamic  combat  environment.  The  major  HF/C^  issues 
associated  with  the  design  of  the  intelligence  analysis  component s)  of 
the  ASAS  thus  deal  with  task  definition,  organization,  tradeoffs  between 
automated,  semi-automated,  and  manual  implementations,  and  training. 

These  issues  cannot  be  addressed  independently;  moreover  the  design 
approaches  eventually  adopted  will  have  significant  impact  on  both  cost 
and  performance. 

HF/C2  KNOWLEDGE  REQUIREMENTS/AVAILABILITY 

The  HF/C^  knowledge  required  to  accomplish  the  design  role  must 
fall  into  a  category  of  system  specific  for  the  intelligence  analysis 
component  cf  ASAS.  The  current  availability  of  knowledge  relative  to  the 
intelligence  analysis  component  is  presented  in  the  following  table. 


—  Continued  -- 
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EXHIBIT  5-4:  SAMPLE  WORKSHEET  -  DESIGN  ROLE 
(Continued) 

Comparison  of  this  table  with  exhibit  5-2  indicates  that  sufficient 
information  exists  on  environmental  effects  for  direct  application 
demonstration.  Knowledge  available  regarding  cognitive,  workstation  and 
organi zati onal  factors  suggests  that  applied  and  exploratory  research 
efforts  will  be  needed  to  complete  the  design. 

TYPE  OF  HF/C2  EFFORT  REQUIRED 

As  is  evident  in  the  preceding  table,  the  performance  of  the  HF/C^ 
role  in  designing  the  intelligence  analysis  component  of  the  ASAS  will 
require  the  initiation  of  a  number  of  research  efforts,  both  applied  and 
exploratory,  to  extend  general  knowledge  and  professional  opinion  to  ASAS 
system  specific  knowledge  prior  to  the  initiation  of  the  design  pnase. 

The  following  potential  projects  would  accomplish  the  necessary 
research.  * 


-  i 

J-The  sample  worksheet  includes  only  two  of  the  potential  applied 
research  projects.  The  actual  worksheet  would  include  all  potential 
projects  --  application  demonstration,  applied  and  exploratory. 


--  Continued  -- 


EXHIBIT  5-4:  SAMPLE  WORKSHEET  -  DESIGN  ROLE 
; Concluded) 

(1)  Applied  Research  Work  Uni t:XXXXXXX 

Project: _ 

Principal  Investigator: _ 

Start  Date:  1  August  1980 

Objective:  To  identify  the  cognitive  factors  associated  with 
intelligence  analysis  and  determine  procedures 
appropriate  to  intelligence  analysis  tasks. 

Approach:  Examination  of  Prototype  TCAC  and  BETA  systems,  in 

the  context  of  general  cognitive  psychology. 

Time  for  First  Use:  1  May  1981,  in  Design. 

(2)  Applied  Research  Work  Unit:  XXXXXXX 

Project:  _ 

Principal  Investigator: _ 

Start  Date:  1  January  1981 

Objective:  To  develop  a  model  of  cognitive  processes  and 

factors  in  intelligence  analysis  tasks  for  use  in 
design  tradeoff  analyses. 

Approach:  General  cognitive  psychology  for  model  design, 

Monte  Carlo  implementation,  interactive  computer 
program. 

Time  for  First  Use:  1  August  1981,  in  Design. 
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5.3  PRESENTATION  OF  PROPOSED  PROGRAM 

Project  worksheets  similar  to  that  presented  in  the  previous  section 
are  the  basic  elements  in  developing  the  systems-oriented  part  of  an  ART 
HF/C2  research  program.  A  large  number  of  such  worksheets  would  be 
developed  to  identify  many  potential  projects  for  consideration  in 
organizing  the  HF/C2  program.  As  noted  earlier,  DA  financial 
guidelines,  system  priorities,  and  other  factors  would  influence  the 
program  composition  proposed  to  DA.  Recognizing  that  the  process 
described  in  this  report  was  designed  to  assist  in  developing  an  HF/C2 
program  that  is  relevant  to  the  Army,  it  also  produces  information  that 
can  be  used  to  portray  the  relevance  of  ARI's  HF/C2  research  program. 
Exhibits  5-5  through  5-12  are  examples  of  the  kinds  of  data  that  can  be 
prepared  for  review  by  the  OA  staff,  using  the  information  inherent  in 
the  framework. 

Exhibit  5-5  would  provide  an  overview  of  all  the  Army's  systems,  by 
functional  system  type  and  their  status  in  the  development  cycle,  that 
were  considered  in  developing  the  ARI  HF/C2  system-oriented  research 
program.  The  influence  that  the  HF/C2  program  is  intended  to  have  on  a 
particular  system,  the  total  cost  by  type  of  relevant  HF/C2  effort,  and 
when  it  is  anticipated  that  the  results  of  the  ARI  efforts  will  have  an 
influence  on  the  system  is  shown  in  exhibit  5-6.  Similar  exhibits  would 
be  available  for  each  system  listed  in  exhibit  5-5.  Exhibits  5-7  through 
5-11  are  intended  to  show  how  the  total  ARI  HF/C2  systems-oriented 
program  costs  breakout  by  functional  system  type,  acouisition  phase  of 
the  influenced  systems,  the  HF/C2  influence  (HF/C2  roles),  type  of 
HF/C2  effort  in  the  research  program,  and  when  the  results  of  the 
program  would  first  be  used  to  influence  the  development  of  Army  systems. 


Exhibit  5-12  depicts  the  structure  of  the  AR I  HF/C2  non-system  program 
by  type  of  activity,  specific  project,  and  the  costs  by  budget  category. 
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EXHIBIT  5-5:  ARI  HF/C2  SYSTEM  PROGRAM 
-  OVERALL  SYSTEMS  RELEVANCE  - 


SYSTEM  ACQUISITION  STATUS 


Function 


System  Tyoe 

Pre-ConceDt 

Concept 

Devel ODment 

Producti on/DeDl oyment 

Intel  1 igence: 

$24 

ASAS 

AGTELIS 

COMFAC 

TCAC 

SOTAS 

MAGI IC 

HAALS 

• 

• 

TACELIS 

• 

• 

Guardrail  V 

• 

Command  Control : 

• 

ECS2 

• 

NBDS 

• 

• 

Electronic  Warfare: 

$27 

TACJAM 

$42 

QuickFix 

• 

Field  Artillery: 

$53 

BSTAR 

• 

BCS 

TACFIRE 

FAMAS 

PADS 

FIST 

CSWS 

• 

• 

GSRS 

• 

• 

ADA: 

$60 

• 

$42 

• 

CCS 

« 

• 

TSQ-73 

• 

• 

Patriot 

Maneuver  Unit: 

$71 

• 

DIV-86 

• 

XM-1 

Infantry  Squad 

• 

Corps-86 
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EXHIBIT  5-6:  ARI  HF/C2  SYSTEM  PROGRAM 
-'RELEVANCE  Tg  SYSTEM  S^  - 


HF/C2  Role 

TvDe  of  HF/C2  Effort 

Cost 

Time  of  First 

Design  Task 

1 

Applied  Research 

$ 

1983 

2 

• 

• 

Application  Demonstration 

• 

• 

S 

1981 

• 

• 

• 

Design  Organization 

• 

Application  Demonstration 

$ 

• 

1983 

Performance  Evaluation 

Exploratory  Research 

$ 

1984 

System  Total  $T 


EXHIBIT  5-7:  AR I  HF/C2  SYSTEM  PROGRAM 
-  COSTS  BY  FUNCTIONAL  SYSTEM  TYPE  - 


Functional  HF/C2 

System  Type  Influence 

Intelligence: 


Type  of 

HF/C2  Effort  Cost 


ASAS 

TCAC 


Functional  Specifications  Applied  Research 

Functional  Requirements  Applied  Demonstration 


Intelligence  Subtotal  Sj 

Command  Control : 


ECS2 

Functional  Specifications 

Applied  Research 

$ 

s13 

• 

• 

Performance  Evaluation 
* 

• 

Exploratory  Research 
• 

• 

s 

• 

Command  Control  Subtotal 

$c 

ectronic 

Warfare: 

T AC JAM 

Design  Task  3 

Appl ied  Research 

$ 

QuickFi x 

Performance  Evaluation 

Application  Demonstration 

s 

Electronic  Warfare  Subtotal 

SEW 

Total 
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EXHIBIT  5-8:  ARI  HF/C2  SYSTEM  PROGPAM 
-  COSTS  BY  SYSTEM  ACQUISITION  PHASE  - 


system 

Acauisition  Pnase 


H" 

>.*’  jence 


*yne  of 
nc  fZ<~  E**orz 


3re-Conceot: 


s 


a 


3unctiona'  3.eou  •  rements 


rjnct'ora'  Soec • • • ;at ■ ons 


Application  Demonstration 
Ado' -ec  Research 


3,-e-Conceoc  Subtotal 


^.onceotua . 
ASAS 
3S*AR 


•‘^ncfona’  Snec  ‘  *  •  cat  'ons 
aes-cr  Acquisition  Tas* 
Oes'cr  >;am;ation 


~oc'-ec  Aesearcr 
A-;' >eo  3esearcr 
Ado  1 ■ ec  Aesea re* 


Concectua I  SuDto: 

Develocment: 

SCTAS 

oe--'ormance  Evaluat’cn 

Application  Demonstration 

Ratriot 

oerf.rmance  Evaluation 

Applied  Research 

Devel ooment  Subtotal 


Producticn/Deployment: 

TACFIRE  Performance  Evaluation  Aoplication  Demonstration 


Product! on/Deplo.vment  Subtotal 

Total 
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EXHIBIT  5-9:  ARI  HF/C2  SYSTEM  PROGRAM 
-  COSTS  BY  HF/C2  ROLE  (INFLUENCE)  - 


HF/C2  Influence 
Functional  Requirements: 


S 

S 

S 


1 

2 

3 


Type  HF/C2  Effort  Cost 


Application  Demonstration  $ 
Exploratory  Research  $ 
Applied  Research  S 


Functional  Requirements  Subtotal  $R 

Functional  Specifications: 

Sg  Applied  Research  $ 

S.J2  Appl  ication  Demonstration  $ 


$ 

Functional  Soecifications  Subtotal  S 

Design  Tasks: 

S^^  Applied  Research  $ 

S-j 2  Applied  Research  $ 


Task  Design  Subtotal 


Total  $T 
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EXHIBIT  5-10:  AR I  HF/C2  SYSTEM  PROGRAM 
-  COST  BY  TYPE  OF  EFFORT  - 


?  ?  Time  of 

Type  of  HF/C^  Effort  HF/C  Influence  First  Use  Cost 


Application  Demonstration: 


S,  Functional  Specifications  1981  $ 

Design  Organization  1982  S 

S.  Design  Situation 

Assessment  Task  7980  S 

Performance  Prediction  1983  S 


Application  Demonstration  Subtotal 


Applied  Research: 

S,  Oesiqn  Task  4  1982 

sio  Performance  Evaluation  1984 

Performance  Evaluation  1983 


$ 


D 


$ 

$ 

$ 


Applied  Research  Subtotal  $A 

Exploratory  Research 

Functional  Requirements  1984  $ 

S ^ Q  Design  Organization  1987  $ 


Exploratory  Research  Subtotal 

Total  $t 
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EXHIBIT  5-11:  ARI  HF/C2  SYSTEM  PROGRAM 

-  COST  BY  FIRST  USE  HORIZON  - 


First  Use  Horizon 


Immediate  (0-3  rears) 
$1 
S3 
^5 


HF/C2 


Infl uence 


Cost 


Design  Task  3  $ 
Performance  Evaluation  $ 
Functional  Requirements  $ 


Midterm  (3-5  Years) 


Immediate  Use  Subtotal  $j 


Design  Organization  $ 
Design  Task  7  $ 
Performance  Evaluation  $ 


Long  Term  (5-10  Years) 


31 

?32 

s33 


Midterm  Use  Subtotal  $M 


Functional  Requirements  $ 
Functional  Specifications  $ 
Design  Organization  $ 


Long  Term  Use  Subtotal  $L 


Total  $T 
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EXHIBIT  5-12:  ARI  HF/C2  NON-SYSTEM  PROGRAM 


Integrate/Document 
HF/C2  System 
Methodology 


B.  Explore  Behavioral 
Science  Technologies- 
for  Application  to  Zc 


Perform  Research  in 
Anticipation  of 
HF/C2  Needs 


Specific  Project 


Sub-total  Non-System 


Cost 

6.1  6.2 


?6. 1  J6.2 


Total  Non-System 


6.0  GENERAL  COMMENTS 


Previous  sections  of  this  report  described  a  framework,  that  could  be 
used  to  identify  human  factors  research  and  analysis  needs  in  the  a, .a  of 
command  and  control.  This  final  section  presents  some  brief  comments, 
somewhat  tangential  to  the  rest  of  the  report,  but  with  a  similar  objec¬ 
tive  --  that  of  identifying  command  and  control  research  and  analysis 
activities  which,  when  conducted  ,  will  improve  the  operational  capabil- 
ites  of  the  Army. 

Anyone  who  has  been  in  the  field  with  operational  units  recognizes 
that  there  is  a  multiplicity  of  command  and  control  problems,  and  that  a 
research  framework  is  not  essential  to  their  identification.  Command  and 
control  problems  exist  within  the  tank  crew,  the  infantry  squad,  the  tank 
platoon,  the  G-2  and  G-3  shops,  the  fire  direction  center,  air  defense 
fire  units,  and  other  Army  systems  that  must  collect  information,  assimi¬ 
late  it,  make  decisions,  and  communicate  the  decisions.  Rather,  the 
difficulties  appear  to  be  that: 

(1)  there  is  no  organized  approach  to  the  identification  of  rele¬ 
vant  HF/C^  analysis  and  research  programs  that  senior  mana¬ 
gers  would  deem  relevant  to  accomplishment  of  Army  missions, 
and 

(2)  there  is  no  vocal  consituency  within  the  Army  that  is  actively 
raising  the  Army's  consciousness  of  existing  and  potential 
command  and  control  problems  and  proposing  efforts  to  develop 
sol  uti  ons. 

The  framework  described  in  this  paper  may  be  useful  in  rectifying  the 
former.  We  believe  ARI's  work  in  the  HF/C^  area  has  perhaps  been  too 
passive  with  respect  to  both  of  the  above  difficulties,  and  accordingly, 
offer  the  following  suggestions  for  consideration: 
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(1)  AR1  must  actively  identify  where  significant  HF/f’  problems 
exist  (or  may  arise  downstream)  with  operational, 
developmental,  and  conceptual  systems.  All  major  Army  systems 
should  be  tracked  continually  regarding  the  status  of  HF 
support  provided  and  needed  (i.e.,  status  of  the  HF/f-'1  role 
dimension).  Problem  identification  emphasis  should  he  placed 
on  identifying  where  the  absence  of  such  support  results  in 
current  or  near- *vrm  problems. 

(?)  The  HFT-1  performance  evaluation  role  must  provide  feedback 
on  the  efficacy  of  the  HF  f'1  desion  efforts.  The  ARI  field 
offices  should  be  the  principal  providers  of  this  feedback  bv 
identifying  HF  '(>’  "lessons  learned"  from  reports  of  previous 
battles  (e.g.,  Sinai  battles)  and  HF/C?  problems  observed  in 
field  exercises.  THis  information  should  be  an  integral  input 
to  development  of  the  HF/C^  research  and  analysis  program. 

(3)  ARI  should  develop  an  organ  iced  and  dynamic  HF  research 
and  analysis  program  using  a  framework  similar  to  that, 
described  in  sections  3.0  to  S.O  of  this  report.  The  program 
should  evolve  continually  and  reflect  its  utility  bv 
documenting  the  Army  systems  it  has  influenced  and  the 
relevance  of  proposed  HF'C1  efforts  to  developmental  and 
conceptual  systems.  This  may  require  the  development  of  a 
"systems  HF  'C1  status"  data  base  (i.e.,  what  has  been  and 
ought  to  be  done)  and  an  "HF  a'/  knowledge  area"  data  base  for 
use  in  both  developing  and  documenting  ARI’s  HF  'XV  program. 


r*  in 
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(4)  Identification  of  command  and  control  problems  by  ARI  and  an 
organized  HF/C^  study  program  is  necessary,  but  not  sufficient, 
to  build  a  viable  HF/C^  program.  ARI  must  actively  work  on 
raising  the  awareness  and  perception  of  Army  managers  that  such 
problems  do  exist,  they  have  associated  effectiveness  (or  cost) 
ramifications,  solutions  can  be  developed  to  resolve  them,  and 
that  ARI  has  an  organized  ongoing  program  to  develop  such 
solutions.  Using  private  sector  vernacular,  ARI  must  "market" 
and  "sell"  the  idea  that  HF/C-  research  and  analysis  is 
relevant,  important,  feasible,  and  useful.  If  possible,  it 
would  be  useful  to  document  a  small  number  of  HF/C^  "success 
stories"  in  the  framework  format. 

(5)  Tne  Army's  physical  science  laboratories  are  continually 
advancing  new  technologies,  either  to  resolve  a  recognized 
problem  or  propose  new  capabilities  for  the  operational  forces. 

In  an  analogous  fashion  (,and  consistent  with  an  active  HF/C^ 
program),  ARI  should  continually  search^-  for  and  advance  new 
behavioral  technology  applications  as  opportunities  for  the 
Army  to  pursue  (e.  g.  ,  self  training  systems). 

(6)  As  noted  earlier  in  this  report,  both  ARI  and  HEL  are  involved 
in  providing  HF/C-  support  for  the  acquisition  of  Army  systems. 
Although  the  distinction  is  not  precisely  defined,  HEL  appears  to 
be  concerned  with  operator/machine  HF  interface  issues  and  ARI's 
human  factors  group  witn  HF/C-  procedural /organizational 
aspects  to  assist  users  (e.  g.  ,  commanders,  G-d,  G-i,  etc.) 

ide  the  defense  area  or  as  part  of  the  non-system  ■•elated  HF/C- 

arch  and  analysis  program. 
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of  Army  systems.  HEL  is  active  in  implementing  their  research 
efforts  to  assist  in  the  development  of  Army  systems.  In  con¬ 
trast,  ARI  appears  to  have  interpreted  its  principal  role 
(described  in  AR  602-1,  AR  70-8,  and  AR  70-1)  as  one  of  devel¬ 
oping  HF/C^  knowledge/methodology  and  demonstrating  it,  but 
not  the  active  use/implementation  of  HF/C^  knowledge/ 
methodology  in  the  development  of  Army  systems.  Because: 

•  we  are  unable  to  identify  an  organization  of  trained  behav¬ 
ioral  analysts  in  the  Army  that  is  using  the  procedural  and 
organizational  HF/C^  knowledge/methodology  to  assist  in  the 
development  of  Army  systems; 

•  senior  managers  in  the  Army  continue  to  stress  the  importance 
of  effective  command  and  control  as  a  major  ingredient  in 
Army  units;  and 

•  Army  regulations  do  not  appear  to  preclude  it; 

we  believe  that  the  ARI  HF/C2  program  should  more  actively 
contribute  to  the  development  of  major  Arny  systems  by 
performing  (or  assisting  in  performing)  the  functional  speci¬ 
fication,  design  (tasks,  organization,  etc.  )  and  performance 
evaluation  activities  described  in  section  2.0  of  this  report. 
This  will  require  more  active  participation  in  both  the 
conceptual  and  validation  phases  of  a  system's  development. 
Paragraph  3-4  of  AR  70-8  explicitly  indicates  that  ARI  should 
participate  in  Advanced  Development  (6.  3A)  activities  which 
"...involve  the  application  of  scientific  knowledge. ..  to 
current  or  potential  field  problems"  (paragraph  1- 7b ). 

Paragraph  3-4f  indicates  that  the  developing  agency  should  turn 
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the  end  product  of  6.  3A  ROTE  activities  over  to  the  using/ 
operating  agency  for  implementation.  However,  because  such 
implementations  of  HF/C^  knowledge  and  methods  do  not  appear 
to  be  occurring,  we  believe  that  (as  suggested  at  the  end  of 
paragraph  3-4f)  ARI  should  more  actively  assist  users  in  the 
implementation  process  to  develop  effective  procedures, 
organizations,  training,  etc.  for  major  systems. 
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APPENDIX:  APPROACHES  TO  EVALUATING  COMMAND- CONTROL  SYSTEMS 

Command-control  systems,  which  may  be  entirely  human  or  mixed 
man-machine  systems,  must  be  evaluated  in  many  ways  during  the 
development,  procurement,  and  use  process.  Evaluators  may  be  concerned 
with  evaluating  the  total  marginal  cost  of  the  systems,  their  logistic 
demands,  their  maintenance  demands,  their  reliability,  their  failure 
modes  and  capability  to  provide  adequate  continuity  of  operations,  the 
training  and  other  personnel  requirements  associated  with  the  system, 
and/or  the  functional  performance  of  the  systems  in  their  major  roles. 
This  appendix  is  concerned  only  with  the  problem  of  evaluations  of 
command  control  systems  with  respect  to  their  functional  performance. 

It  discusses  some  of  the  major  issues  related  to  the  choice  of  an 
approach  to  the  evaluation  problem.  It  does  not  address  the  detailed 
problems  of  implementation  of  specific  approaches;  there  is  a  sizeable 
literature  already  in  existence  addressing  the  evaluation  and  analysis 
of  human  or  man-machine  information  processing  systems,  and  this 
appendix  cannot  provide  coverage  of  the  voluminous  material  which  has 
been  published  on  the  detailed  problems  in  the  evaluation  process. 
Rather,  it  provides  an  overview  of  issues  which  bear  on  the  choice  of  an 
overall  approach  to  an  evaluation.  These  issues  will  be  discussed  in 
terms  of  major  dimensions  of  evaluation  approaches,  where  an  approach 
can  be  defined  in  terms  of  the  choices  made  in  terms  of  these 


dimensions. 


Nine  of  these  major  dimensions  will  be  discussed  in  this  appendix. 


They  involve: 

(1)  the  contrast  between  unitary  and  partial  approaches  to  system 
evaluation, 

(2)  the  dimension  running  from  the  use  of  measures  of  military 
effectiveness  to  the  use  of  system  performance  measures, 

(3)  the  distinction  between  approaches  which  involve  the 
evaluation  of  a  command-control  system  on  individual  tasks  and 
those  which  evaluate  a  system  on  a  realistic  stream  of  tasks, 

(4)  the  contrast  between  those  approaches  intended  to  provide  a 
quality  assessment  from  those  intended  to  provide  diagnostic 
information  for  system  design,  redesign,  or  optimal  use, 

(5)  the  distinction  between  approaches  which  involve  an  absolute 
evaluation  of  one  or  more  systems  and  those  which  involve  only 
comparative  evaluation  of  two  or  more  systems, 

(6)  the  dimension  extending  from  approaches  involving  evaluation 
during  real  system  use  to  approaches  involving  evaluation 
based  on  theoretical  models  of  the  system, 

(7)  the  distinction  between  observation-  or  measurement -based 
evaluations  and  those  based  on  participant  reporting, 

(8)  the  contrast  between  objectified  and  judgmental  evaluation 
techniques,  and 

(9)  the  dimension  which  describes  the  detailed  indices  being  used 
the  the  evaluation. 

The  first  of  these  dimensions  which  is  discussed  contrasts  unitary 
and  partial  approaches  to  command-control  system  evaluations.  In  a 
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unitary  approach,  the  system  is  examined  or  modelled  in  its  entirety  (or 
very  nearly  so);  in  a  partial  approach,  the  system  is  divided  into 
functionally  or  physically  separated  subsystems,  and  these  are  evaluated 
separately.  A  compound  approach  is  also  possible,  in  which  the  system 
is  experimented  with  or  modelled  in  entirety,  but  measurements  are  taken 
and  examined  in  ways  designed  to  estimate  the  performance  of  the 
subsystems. 

Generally,  the  partial  approach  is  less  expensive  to  implement  but 
leaves  serious  risks  of  misevaluation  of  the  total  system  through 
neglect  of  unforeseen  ineractions  among  subsystems.  It  is  also 
sometimes  difficult  to  identify  appropriate  subsystems  for  separable 
evaluations.  The  unitary  approach  does  not  involve  the  identification 
of  subsystems  and  separate  subfunctions,  but  also  cannot  be  used  to 
provide  information  which  will  help  system  designers  improve  the  various 
elements  of  the  system  separately.  The  compound  unitary-multi  part 
approach  requires  probably  the  greatest  effort,  both  preparatory  (in 
terms  of  identifying  and  analyzing  subsystems  and  subfunctions)  and  in 
accomplishment  (in  terms  of  the  expenses  of  experiment  or  simulation). 

Examples  of  the  types  of  approaches  distinguished  here  may  be  seen 
in  a  hypothetical  evaluation  of  an  artillery  command-control  system.  A 
unitary  approach  would  involve  experimentation  and/or  simulation  of  the 
entire  multi-person,  multi-machine  system  in  an  appropriate  environment 
and  the  estimation  of  overall  performance  measures.  These  measures 
might  include  such  scales  as  fraction  of  targets  attacked,  predicted 
casualties  caused,  etc.  Partial  approaches  might  separate  several  of 
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the  types  of  functions  being  performed  (e. g.  ,  storage  and  retrieval  of 
target  information,  selection  of  targets  for  fire,  computation  of 
parameters  for  fire  missions  against  targets,  communication  of  missions 
to  batteries).  Separate  models  or  experiments  could  then  be  used  to 
analyze  each  of  these  functions  (or  a  different  set  of  subfunctions  or 
subsystems,  e.  g.  ,  a  separation  of  the  human  portion  of  the  system  from 
the  machine  portion)  in  terms  of  individual  performance  measures  such  as 
time  required  to  enter  target  information,  probability  of  timely 
retrieval  of  target  information,  fractions  of  high-priority  and 
low-priority  targets  fired  on,  appropri ateness  of  fire  mission 
computations  for  selected  targets,  communication  delays  between  fire 
mission  computations  and  battery  receipt  of  firing  order,  etc.  A 
compound  approach  would  involve  aspects  of  both  methods. 

Another  dimension  which  describes  alternative  approaches  to  the 
evaluation  of  command-control  systems  is  the  dimension  which  runs 
between  extremes  one  of  which  uses  measures  addressing  military  utility 
and  the  other  of  which  uses  measures  of  system  performance.  At  the 
extreme  level,  military  utility  measures  address  an  estimate  of  overall 
force  effectiveness:  the  lowest  level  of  system  performance  measures 
address  such  micro-issues  as  the  average  time  between  a  query  or 
directive  to  a  machine  component  and  the  response  or  accomplishment. 
Although  often  associated  in  practice  with  the  unitary-partial 
dimension,  this  dimension  is  actually  a  distinct  area.  It  is  possible 
to  have  military  utility  measures  in  a  partial  approach  or  system 
performance  measures  in  a  unitary  approach,  as  well  as  the  more  usual 
patterns.  Examples  in  an  air-defense  command-control  area  might  be  the 
comparison  of  two  target  selection  algorithms  in  terms  of  the  predicted 
mean  number  of  penetrating  aircraft  (considered  as  part  of  a  specified 
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raid)  which  would  deliver  their  ordnance  on  a  target,  or  the  comparison 
of  two  overall  systems  in  terms  of  the  mean  range  at  which  target 
aircraft  were  first  engaged. 

This  dimension  is  not  binary,  and  in  some  cases  it  is  not  clear 
where  some  measures  fall.  (The  mean  range  at  first  engagement  measure 
was  used  above  as  an  example  of  a  system  performance  measure  --  to  some 
analysts,  this  might  be  considered  a  military  worth  measure.)  Even 
though  the  dimension  is  neither  binary  nor  perfectly  defined,  it  is  a 
useful  dimension  for  distinguishing  approaches.  In  choosing  an 
approach,  the  use  of  either  extreme  should  be  avoided.  Measures 
addressing  very  high-level  military  utility  provide  little  diagnostic 
information  to  system  analysts  and  designers  and  are  highly  dependent  on 
assumptions,  models  of  various  effects,  and  environmental  conditions. 
System  performance  measures  alone  may  be  seriously  misleading  in  the 
frequent  cases  in  which  military  utility  is  little  effected  (or  effected 
in  a  highly  non-linear  pattern)  by  the  system  performance  of  concern. 

An  approach  using  middle-level  measures  or  a  package  of  measures 
including  both  system-level  and  military-utility  level  scales  is 
preferable  to  either  extreme  alone. 

Both  these  first  two  dimensions  have  been  different  ways  of 
describing  the  degree  of  holism  of  an  approach.  There  is  a  third 
dimension  which  also  concerns  the  degree  to  which  an  approach  is 
holistic,  but  in  yet  a  further  distinct  way.  This  third  dimension 
involves  the  distinction  between  the  assessment  of  command-control 
systems  on  individual  tasks  and  their  assessment  on  a  realistic  stream 
of  tasks.  A  fire-control  system  might  be  assessed,  for  example,  on  a 
problem  defined  in  terms  of  a  single  target  and  associated  fire  mission 
(or  a  single  group  of  targets  and  multiple  fire  missions),  or  in  terms 
of  a  realistic  time-line  of  activities  including  both  periods  with  few 
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activities  and  other  periods  with  single  and  multiple  tasks  overlapping 
in  realistic  ways.  This  type  of  evaluation  addresses  not  only  task 
performance  on  well-defined  tasks  but  also  the  capability  of  the 
individual  or  staff  organization,  with  whatever  machine  support  is 
provided,  to  appropriately  structure  tasks  in  an  unstructured 
environment.  To  some  extent,  this  dimension  associates  with  the 
distinctions  often  made  between  "free-play"  exercises  and  evaluations 
and  "structured"  exercises.  For  most  command  and  control  systems,  it  is 
critical  that  at  some  points  both  early  and  late  in  their  development 
cycle  they  be  evaluated  (and  designed  to  handle)  the  realistic, 
unstructured  stream  of  tasks  representing  the  situations  in  which  they 
will  actually  be  employed.  On  the  other  hand,  throughout  the  middle  of 
their  development  life,  structured  evaluations  generally  provide  clearer 
feedback  to  the  developers  and  designers  on  development  and  design 
issues,  and  are  therefore  preferable.  To  some  extent,  this  same  general 
tendency  is  observable  in  all  of  the  three  holism  dimensions  described 
here.  Unitary  and  military  utility  approaches  are  more  useful  near  the 
beginning  and  end  of  development  programs,  while  partial  and 
performance-oriented  approaches  provide  better  detailed  guidance  to 
designers  and  developers  during  the  middle  of  the  development  process. 

These  observations  concerning  the  differential  utility  of  more  and 
less  holistic  approaches  at  different  phases  of  development  in  fact 
bring  out  another  dimension  on  which  evaluation  approaches  may  be 
contrasted.  This  dimension  involves  distingushing  evaluations  intended 
to  serve  in  a  quality-assessment  role  from  those  assessments  which  are 
intended  to  provide  diagnostic  information  suitable  to  guide  designer, 
developer,  or  user  actions  into  the  most  profitable  channels.  Quality 
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assessments  of  designs  are  generally  used  in  development  processes  at 
the  very  early,  concept  definition  phases,  so  that  potential  overall 
impact  may  be  assessed  before  commiting  even  to  the  development  process, 
and  at  the  late  stages,  when  the  system  is  in  a  well -developed  state  and 
one  wishes  to  test  the  degree  to  which  early  expectations  have  been  met 
so  that  a  reasoned  decision  on  procurement  can  be  made.  (In  the  current 
Army  development  process,  quality  assessments  are  also  associated  with 
certain  mid-cycle  decision  points.) 

Diagnostic  examinations  are  useful  in  the  early,  concept  definition 
phases  and  throughout  the  middle  of  the  development  process.  In  the 
early  phases,  diagnostic  examinations  provide  information  on  design 
directions  in  what  is  at  that  point  still  a  vaguely-defined  product. 
After  a  commitment  to  development  along  the  lines  of  a  particular 
design,  diagnostic  evaluation  permits  the  examination  of  alternative 
design  approaches  within  the  general  design  selected  in  the  early 
phases. 

Another  dimension  on  which  approaches  may  be  contrasted,  still  in 
the  general  area  of  the  intention  of  the  evaluation,  is  that  which 
contrasts  absolute  evaluations  with  comparative  evaluations.  Absolute 
evaluations  are  generally  used  to  examine  the  performance  of  systems  in 
terms  of  measures  or  criteria  which  are  stated  in  a  fashion  making  no 
reference  to  alternative  systems.  Comparative  evaluations  involve 
inherent  comparisons  with  alternative  systems.  Thus,  for  example,  an 
evaluation  of  a  command  center  support  system  against  the  standard  "it 
must  allow  performance  of  all  functions  with  20%  fewer  staff  than  the 
present  system"  is  inherently  comparative.  In  choosing  an  evaluation 
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approach,  the  principal  concern  in  this  area  is  that  the  approach 
correspond  to  the  actual  intentions  of  those  who  will  be  using  the 
results  of  the  evaluation. 

The  remaining  dimensions  have  to  do  with  the  types  of  methods  and 
measurements  used  in  evaluations  rather  than  with  the  degree  of  holism 
or  the  intentions  of  the  evaluation.  The  first  of  these  remaining 
dimensions  is  that  which  contrasts,  at  one  extreme  of  a  continuous 
scale,  evaluations  conducted  on  systems  during  real  use  with,  at  the 
other  extreme,  evaluations  performed  by  theoretical  modelling  and 
inference.  In  the  area  of  developmental  command  and  control  systems, 
making  observations  of  a  system  in  real  use  is  generally  impossible. 

The  closest  achievable  case  is  usually  either  evaluation  in  field  trials 
or  in  field  experiments,  possibly  with  simulations  of  some  or  all  of  the 
environment  of  the  system.  There  are  many  discussions  available  which 
address  the  relative  values  of  approaches  on  different  points  of  this 
dimension,  which  includes  laboratory  testing,  detailed  simulation,  and 
completely  theoretical ly-based  evaluations  at  the  more  abstract  end  of 
the  scale  ,  and  the  arguments  for  and  against  these  techniques  are 
basically  the  same  in  the  command-control  systems  area  as  in  other 
areas.  Accordingly,  this  report  will  not  contain  any  further  discussion 
on  these  topics. 

Another  dimension,  which  is  more  closely  related  to  the 
command-control  area,  contrasts  approaches  which  use  observation  and 
measurement  techniques  from  those  which  rely  on  participant  reporting  on 
a  system.  In  this  area  as  in  most,  observational  techniques,  in  which 
the  evaluations  of  a  system  are  made  by  persons  independent  of  the 
operations,  are  generally  preferred  for  their  objectivity.  Another 
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dimension  which  is  often  confused  with  this  one,  but  is  in  fact  a 
distinct  dimension  of  evaluation  approaches,  is  that  which  contrasts 
objectified  evaluations  with  judgmental  ones.  Judgmental  observations 
made  by  an  observer  are  significantly  different  than  judgmental 
observations  made  by  a  participant  in  a  command-control  system. 

Further,  particpant  observations  may  be  used  even  when  objectified 
measures  are  used.  As  with  other  dimensions,  this  objectified- 
judgmental  dimension  is  not  binary,  but  has  middle  areas  in  which 
objectified  scales  which  are  simply  formal  representations  of  judgment 
are  used.  Generally,  objectified  methods  are  preferred,  although 
judgmental  techniques  may  often  shed  diagnostic  light  on  possible  design 
changes  or  unforeseen  possible  uses  (or  detriments)  of  a  system. 

The  last  dimension  discussed  here(and  perhaps  the  most  important) 
is  the  dimension  which  describes  the  measurement  indicies  which  are 
used  in  the  evaluation.  It  is  not  possible  here  to  describe  all,  or 
even  a  major  fraction  of  the  possible  indices  in  detail.  Instead,  they 
are  described  in  terms  of  three  general  types  of  index:  those  which  deal 
with  timing  aspects  of  the  system  performance,  those  which  deal  with 
procedural  aspects  of  performance,  and  those  which  deal  with  substantive 
measurements  of  performance.  (Indices  which  combine  these  aspects  are 
also  possible,  of  course.  ) 

Indices  which  deal  with  timing  aspects,  such  as  delays  or 
measurements  of  the  time  required  to  perform  certain  tasks,  are  easily 
recognized.  Their  overall  usefulness  depends  on  the  degree  to  which  the 
eventual  military  utility  of  the  system  depends  on  the  timeliness  of  the 
task  performance  (and  the  degree  to  which  alternative  systems  show 


variations  in  timeliness). 

Examples  of  indices  which  deal  with  procedural  aspects  are  such 
measures  as  the  fraction  of  users  who  issued  proper  orders  for  fire 
missions,  or  included  all  necessary  information  in  messages  calling  for 
close  air  support,  or  the  percentage  of  brigades  for  which  situation 
information  is  properly  stored,  etc.  These  indices  are  most  useful  in 
evaluating  command-control  support  systems  which  are  designed  to  reduce 
errors  and  confusion  in  operations,  or  to  make  staff  tasks  easier.  For 
systems  with  clear  decision  support  roles,  it  is  generally  more  useful 
to  use  substantive  indices  of  the  appropriateness  of  the  decisions  made. 

Substantive  indices  involve  attempts  to  measure  the  functional 
performance  of  the  system  in  more  direct  ways.  Typical  indices  in  this 
approach  might  include  the  fraction  of  identifiable  targets  which  were 
identified,  or  the  fraction  of  retrieved  reports  which  were  in  fact 
relevant  to  a  request,  or  the  fraction  of  relevant  reports  which  were 
retrieved,  or  the  fraction  of  critical  decisions  which  an  independent 
panel  reviewing  a  reconstruct  on  of  an  exercise  or  experiment  considers 
to  have  been  significantly  wrong  decisions,  etc.  These  indices  are 
generally  important  in  the  evaluation  of  decision-support  systems, 
although  timing  and  procedural  indices  may  be  equally  critical  even  in 


these  contexts. 


